Shell script

Intro

Steve Bourne wrote the Bourne shell which appeared in the Seventh Edition Bell Labs Research version of Unix. Many other shells have been written; this particular tutorial concentrates on the Bourne and the Bourne Again shells. Other shells include the Korn Shell (ksh), the C Shell (csh), and variations such as tcsh. This tutorial does not cover those shells.

Philosophy

Shell script programming has a bit of a bad press amongst some Unix systems administrators. This is normally because of one of two things:

  • The speed at which an interpreted program will run as compared to a C program, or even an interpreted Perl program.
  • Since it is easy to write a simple batch-job type shell script, there are a lot of poor quality shell scripts around.

Variables

MY_MESSAGE="SOME STRING HERE"
MY_NUMBER=30

Color variables.

# Colors
ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"
COL_YELLOW=$ESC_SEQ"33;01m"
COL_BLUE=$ESC_SEQ"34;01m"
COL_MAGENTA=$ESC_SEQ"35;01m"
COL_CYAN=$ESC_SEQ"36;01m"

# Use
echo -e "$COL_RED This is red $COL_RESET"
echo -e "$COL_BLUE This is blue $COL_RESET"
echo -e "$COL_YELLOW This is yellow $COL_RESET"

There are a set of variables which are set for you already, and most of these cannot have values assigned to them. These can contain useful information, which can be used by the script to know about the environment in which it is running.

The first set of variables we will look at are $0 .. $9 and $#. The variable $0 is the basename of the program as it was called. $1 .. $9 are the first 9 additional parameters the script was called with. The variable $@ is all parameters $1 .. whatever.

The variable $*, is similar, but does not preserve any whitespace, and quoting, so “File with spaces” becomes “File” “with” “spaces”. This is similar to the echo stuff we looked at in A First Script.

As a general rule, use $@ and avoid $*. $# is the number of parameters the script was called with. Let's take an example script: 

echo "I was called with $# parameters"
echo "My name is $0"
echo "My first parameter is $1"
echo "My second parameter is $2"
echo "All parameters are $@"

Concatenate

foo=sun
echo $fooshine     # $fooshine is undefined
echo ${foo}shine   # displays the word "sunshine"

Wildcards

cp /tmp/a/* /tmp/b/
mv /tmp/b/* /tmp/c

Escape characters

echo "Hello   \"World\""

Lopps

Search and remove files with find.

find /home/user -type f -name "*.h" -exec rm -f {} \;

For loop.

for i in 1 2 3 4 5
do
  echo "Looping ... number $i"
done

While loop.

while :
do
  echo "Please type something in (^C to quit)"
  read INPUT_STRING
  echo "You typed: $INPUT_STRING"
done

Another while loop.

while read input_text
do
  case $input_text in
        hello)          echo English    ;;
        howdy)          echo American   ;;
        gday)           echo Australian ;;
        bonjour)        echo French     ;;
        "guten tag")    echo German     ;;
        *)              echo Unknown Language: $input_text
                ;;
   esac
done < myfile.txt

Conditional sentences

CMD Meaning
-z STRING Empty string
-n STRING Not empty string
STRING == STRING Equal
STRING != STRING Not Equal
NUM -eq NUM Equal
NUM -ne NUM Not equal
NUM -lt NUM Less than
NUM -le NUM Less than or equal
NUM -gt NUM Greater than
NUM -ge NUM Greater than or equal
STRING =~ STRING Regexp
1) Numeric conditions
-o noclobber If OPTIONNAME is enabled
! EXPR Not
X && Y And
| Y Or

File conditional.

CMD Meaning
-e FILE Exists
-r FILE Readable
-h FILE Symlink
-d FILE Directory
-w FILE Writable
-s FILE Size is > 0 bytes
-f FILE File
-x FILE Executable
FILE1 -nt FILE2 1 is more recent than 2
FILE1 -ot FILE2 2 is more recent than 1
FILE1 -ef FILE2 Same files

Conditional 1 .

if [$foo == "bar" ]

Conditional 2.

if SPACE [ SPACE "$foo" SPACE = SPACE "bar" SPACE ]

Conditional 3.

if [ ... ]
then
  # if-code
else
  # else-code
fi

Case switch

echo "Please talk to me ..."
while :
do
  read INPUT_STRING
  case $INPUT_STRING in
    hello)
        echo "Hello yourself!"
        ;;
    bye)
        echo "See you again!"
        break
        ;;
    *)
        echo "Sorry, I don't understand"
        ;;
  esac
done
echo 
echo "That's all folks!"

External programs

grep

grep "^${USER}:" /etc/passwd | cut -d: -f5

cat and awk

cat /usr/home/frederico/.cache/defaultgw.log | awk 'getline -1 { print $3 }'

Functions

# vars
time=$(date +"%d/%m/%Y %H:%M:%S")
cpu_temp_log=$(sysctl dev.pchtherm.0.temperature | awk '{ print $2 }' | cut -b 1-4)

# get the cup 0 temp
function get_cpu_temp() {
    echo "$time $cpu_temp_log" >> /usr/home/frederico/.cache/cpu-temp.log;
    cat /usr/home/frederico/.cache/cpu-temp.log | awk 'getline -1 { print " Hammer: " $3 "C " }';
}

# print the cached add
get_cpu_temp

Typical shellscript

List all host neighbors.

#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# https://cronitor.link/p/013b6eac87634372aca2ff3a01d0a886/sUtC6D
#

# vars
ltime=$(date +"%m/%m/%Y %H:%M:%S")
time=$(date +"%m-%m-%Y_%H-%M-%S")
neighborhood=$(nmap -sP 172.16.0.0/24 | awk 'NF==6 {print $6 "\t" $5}')

# find the neighbors
function find_neighborhood() {
    echo "$neighborhood" > /usr/home/frederico/.cache/neighborhood;
    touch /usr/home/frederico/.cache/neighbor/neighborhood-$time.log;
    cat /usr/home/frederico/.cache/neighborhood > /usr/home/frederico/.cache/neighbor/neighborhood-$time.log;
}

# run this mothafucka
find_neighborhood

Returns the IPV4 IP address of the current machine.

#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# https://cronitor.link/p/013b6eac87634372aca2ff3a01d0a886/HisIMq
#

# vars
int_if="re0"
ltime=$(date +"%d/%m/%Y %H:%M:%S")
update_addr=$(ifconfig $int_if | awk '/inet/ {print $2}')

# get ip address
function get_addr() {
    echo "$ltime $update_addr" >> /usr/home/frederico/.cache/ip-addr.log;
    cat /usr/home/frederico/.cache/ip-addr.log | awk 'getline -1 { print $3 }';
}

# print the cached add
get_addr

Returns the current machine OS name.

#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# 
#

# vars
os=$(sysctl kern.ostype | awk '{ print $2 }')
release=$(sysctl kern.osrelease | awk '{ print $2 }')
cpu=$(sysctl hw.model | awk '{ print $5 " " $4  }')
mem=$(sysctl hw.physmem | awk '{ print $2/1e9 "G" }')

function run_tha_shit() {
    echo " $os - $release - $cpu $mem "
}

# run tha shit
run_tha_shit
````

__Returns the current amount of free memory (RAM)__.

```bash
#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# https://cronitor.link/p/013b6eac87634372aca2ff3a01d0a886/HisIMq
#

# vars
ltime=$(date +"%d/%m/%Y %H:%M:%S")
free=$(sysctl vm.kmem_map_free | awk '{ print $2/1e9 }')

# free memory
function free_mem() {
    echo "$ltime $free" >> /usr/home/frederico/.cache/free-mem.log
    cat /usr/home/frederico/.cache/free-mem.log | awk 'getline -1 { print $3 }'
}

# run tha shit
free_mem

Returns the network default route.

#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# https://cronitor.link/p/013b6eac87634372aca2ff3a01d0a886/jv3IpI
#

# vars
ltime=$(date +"%d/%m/%Y %H:%M:%S")
default_gw=$(netstat -n -r -4 | awk '/default/ { print $2 }')

# find the local network default gw
function find_default_gw() {
    echo "$ltime $default_gw" >> /usr/home/frederico/.cache/defaultgw.log;
    cat /usr/home/frederico/.cache/defaultgw.log | awk 'getline -1 { print $3 }'
}

# run tha shit!
find_default_gw

Returns the cpu core temperature.

#!/usr/bin/env bash
#
# Frederico Sales
# <frederico.sales@engenharia.ufjf.br>
# 2022
#
# https://cronitor.link/p/013b6eac87634372aca2ff3a01d0a886/8yUpvT
#

# vars
time=$(date +"%d/%m/%Y %H:%M:%S")
cpu_temp_log=$(sysctl dev.pchtherm.0.temperature | awk '{ print $2 }' | cut -b 1-4)

# get the cup 0 temp
function get_cpu_temp() {
    echo "$time $cpu_temp_log" >> /usr/home/frederico/.cache/cpu-temp.log;
    cat /usr/home/frederico/.cache/cpu-temp.log | awk 'getline -1 { print " Hammer: " $3 "C " }';
}

# print the cached add
get_cpu_temp

Advanced stuff

Declaring arrays.

declare -a ARRAYNAME

Using.

declare -a index_array
index_array=(1 2 3 4 5 6 7 8 9 0)

for i in ${index_array[@]}
do
        echo $i
done

Multidimensional array

read -p "Enter the size of matrix: " n
c=`expr $n - 1`

declare -A arr

# Get the matrix elements

for ((i=0;i<=c;i++))
do        
    for ((j=0;j<=c;j++))
        do
            read -p "enter the value of $i, $j element " arr[$i,$j]
        done
done

# Print the matrix
for ((i=0;i<=c;i++))
do
    for ((j=0;j<=c;j++))
    do
        echo -n "${arr[$i,$j]} "
    done
    echo
done

sysctl

Sysctl is part of Kernel tunable. They are used to customize the behavior of unix like systems at boot, or on demand while the system is running. Some hardware parameters are specified at boot time only and cannot be altered once the system is running, most however, can be altered as required and set permanent for the next boot. The best way to know it features is dump the options into a txt file and analyze it, like so. 

sysctl -a > kernel_tunable.txt

Dump variables into console.

sysctl -a | grep memory
real memory  = 4294967296 (4096 MB)
avail memory = 3826421760 (3649 MB)
...

Writing variables permanently with sysctl

sysctl -w <tunable class>.<tunable>=<value> >> /etc/sysctl.conf

Controllable tunable

class subsystem
abi Execution domains and personalities
crypto Cryptographic interfaces
debug Kernel debugging interfaces
dev Device specific information
fs Global and specific filesystem tunable
kernel Global kernel tunable
net Network tunable
sumrpc Sun Remote Procedure Call (NFS)
user User Namespace limits
vm Tuning and management of memory, buffer, and cache

Further info.

man sysctl

brace expansion

CMD Exit
echo {AB}.js AB.js
echo A B
echo {A,B}.js A.js B.js
echo 1 2 3 4 5
echo ${name} “John”
echo ${name/J/j} “john” (substitution)
echo ${name:0:2} “Jo” (slicing)
echo ${name::2} “Jo” (slicing)
echo ${name::-1} “Joh” (slicing)
echo ${name:(-1)} “n” (slicing from right)
echo ${name:(-2):1} “h” (slicing from right)
echo ${food:-Cake} $food or “Cake”
echo ${STR%.cpp} /path/to/foo
echo ${STR%.cpp}.o /path/to/foo.o
echo ${STR%/*} /path/to
echo ${STR##*.} cpp (extension)
echo ${STR##*/} foo.cpp (basepath)
echo ${STR#*/} path/to/foo.cpp
echo ${STR##*/} foo.cpp
echo ${STR/foo/bar} /path/to/bar.cpp
echo ${STR:6:5} “world”
echo ${STR:-5:5} “world”
BASE=${SRC##*/} “foo.cpp” (basepath)
DIR=${SRC%$BASE} “/path/to/” (dirpath)

Arguments

Symbol Meaning
$# Number of arguments
$* All arguments
$@ All arguments, starting from first
$1 First argument
$_ Last argument of the previous command

TODO

  • dealing with signals
  • chroot
  • patching and diffing
  • calculations with bc
  • persistent background processes
  • recursive search and replacing
  • cheat sheet

About

Author